Skip to content

test(ai): add IPv6 + redirect-canonicalisation localOnlyFetch tests#543

Merged
sroussey merged 8 commits into
mainfrom
claude/beautiful-mayer-gdhUE
Jun 4, 2026
Merged

test(ai): add IPv6 + redirect-canonicalisation localOnlyFetch tests#543
sroussey merged 8 commits into
mainfrom
claude/beautiful-mayer-gdhUE

Conversation

@sroussey
Copy link
Copy Markdown
Collaborator

@sroussey sroussey commented Jun 2, 2026

Summary

This PR adds 4 IPv6 + redirect-canonicalisation regression tests to packages/test/src/test/ai-provider-api/localOnlyFetch.test.ts, on top of what already landed on main via #542 and #544.

The original PR consolidated two plans (Plan A — localOnlyFetch SSRF hardening, and Plan B — ServerCredentialStore hardening). Both plans were merged separately:

After rebasing onto current main, the only net-new content here is the 4 test cases listed below.

What's in the test file diff (44 added lines, 1 file)

All four tests target localOnlyFetch and cover cases not already exercised by main's suite:

  1. accepts a bracketed IPv6 loopback initial URLhttp://[::1]:8080/ (positive case; extractRawHost strips the brackets so isLoopbackHostname sees ::1).
  2. rejects an IPv6 loopback with a zone IDhttp://[::1%25eth0]/ (negative; parseIpv6 rejects any host containing %).
  3. follows a redirect whose Location canonicalizes to a loopback IPv4Location: http://0x7f.0.0.1/. Pins the documented behaviour: extractRawHost(next.href) returns the WHATWG-canonical 127.0.0.1, which IS a loopback literal, so the redirect is followed. The security invariant (never leave the loopback host) still holds — the final destination is 127.0.0.1. Inline comment flags this as the deliberate trade-off so a future pre-canonical redirect validator would flip this test to a reject.
  4. follows a redirect to a bracketed IPv6 loopbackLocation: http://[::1]/ (positive).

Branch history note

The rebase could not be force-pushed from this environment (no GitHub push credential available — only the GitHub MCP API). Instead the branch was rebased via:

  1. A reset commit that brought localOnlyFetch.test.ts back to byte-for-byte match main's version, eliminating the only conflicting file.
  2. A "Update branch" merge of current main into the branch (now succeeds with zero conflicts because every other modified file on the original branch already matched main exactly after fix(ai): close WHATWG canonicalisation bypass in localOnlyFetch (sec) #542 and fix(storage): restore ICredentialStore contract on ServerCredentialStore (sec) #544 landed).
  3. A final commit that adds the 4 new tests on top of the merged tree.

The PR diff vs current main is therefore clean — only the 4 new test cases. The commit history retains the merge commit and the two corrective commits rather than appearing as a single clean rebase.

Test plan

  • bunx vitest run packages/test/src/test/ai-provider-api/localOnlyFetch.test.ts — 19/19 pass (15 from main + 4 new).
  • CI: full lint / typecheck / build.

https://claude.ai/code/session_01Wws8oZpB5imjKL2e7DRXtc

sroussey added 2 commits June 2, 2026 01:31
localOnlyFetch previously validated url.hostname after the WHATWG URL
parser canonicalises it, so non-standard IPv4 spellings (0x7f.0.0.1,
2130706433, 010.0.0.1) silently slipped past the strict-literal loopback
grammar in isLoopbackHostname. Switch both gates - the initial URL and
every redirect hop - to validate the literal host extracted via
extractRawHost, taken from the raw URL string BEFORE canonicalisation.
The fallback to url.hostname is kept as defence in depth.

Adds 7 unit tests covering hex/decimal/octal IPv4 spellings, bracketed
IPv6 literal [::1], IPv6 zone identifier rejection, and the redirect
canonicalisation pin-down behaviour (a Location: http://0x7f.0.0.1/
sent from a loopback origin is followed because next.href resolves to
127.0.0.1, which IS loopback).

Supersedes #542.

https://claude.ai/code/session_01Wws8oZpB5imjKL2e7DRXtc
…es, orphan reasons)

Three classes of issue addressed:

1. Key-injection scope escape.
   userId, projectId, and every credential key are now matched against a
   strict SAFE_SEGMENT grammar (^[A-Za-z0-9_-]{1,128}$). Without this, a key
   like "../../other-user/other-project/leaked" could be smuggled through
   vaultId() and collide with another scope's vault id. Failures are surfaced
   as TypeError without echoing the invalid value back.

2. Delete races and orphan-overwrite.
   - delete() is refactored onto an internal deleteById() helper that
     deletes metadata FIRST, then vault. A throw on metadata.delete leaves
     the entry fully intact and readable; a throw on vault.deleteSecret
     reinstates a pending orphan marker (with orphanReason
     "vault-delete-failed") so readers see absence and operators can
     reconcile, AND surfaces a wrapped error whose message distinguishes
     "orphan marker persisted" from "orphan marker also failed to persist".
   - deleteById skips rows whose metadata is pending: true. This closes
     both the orphan-overwrite path (delete on an existing orphan marker
     used to nuke the vault entry it was tracking) and the concurrent
     put(new)+delete race (delete during the pending window of a new-entry
     put would wipe the vault entry mid-write).
   - get()/has() wrap expiry self-eviction in try/catch so a transient
     KV/vault failure during eviction does not surface as a thrown read.
   - deleteAll() collects per-row failures and throws AggregateError so a
     single bad row no longer silently masks deletion of the rest.

3. Orphan reason discriminator.
   CredentialMetadataRow gains an optional orphanReason field
   ("vault-write-failed" | "metadata-commit-failed" | "vault-delete-failed")
   and a corresponding exported OrphanReason type. Both put() orphan
   paths and the new delete() orphan path stamp the reason so operator
   tooling can pick a remediation strategy without re-running the original
   write. Legacy rows without the discriminator stay invisible to readers
   (covered by an explicit test).

14 new tests added; existing delete-removes-both test upgraded to assert
metadata-first ordering via spy call-order; existing put orphan tests
upgraded to assert orphanReason.

Supersedes #541.

https://claude.ai/code/session_01Wws8oZpB5imjKL2e7DRXtc
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 2, 2026

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 61.93% 24798 / 40036
🔵 Statements 61.79% 25654 / 41518
🔵 Functions 62.91% 4679 / 7437
🔵 Branches 50.58% 12148 / 24014
File CoverageNo changed files found.
Generated in workflow #2498 for commit edc9a05 by the Vitest Coverage Report Action

sroussey pushed a commit that referenced this pull request Jun 3, 2026
…ore (sec)

Follow-up to #543. Two contract regressions discovered in code review:

H1: deleteAll() silently skipped pending/orphan rows because it now
    delegates to deleteById (added in #543) which short-circuits when
    pending=true. Result: no public API cleared orphan markers; a
    "wipe scope" call silently leaked the very rows operators need
    to clean up. Adds forceDeleteById that bypasses the pending-skip;
    deleteAll uses it. deleteById (single-key) unchanged.

H2: get/has/delete now threw TypeError for invalid keys because they
    all routed through vaultId(). ICredentialStore documents these as
    returning undefined/false on missing keys; throwing breaks
    substitutability with InMemoryCredentialStore / EncryptedKvCredentialStore.
    Adds isSafeKey() predicate; readers short-circuit; put() still
    throws via vaultId (only path that persists a colliding id).

M1: SAFE_SEGMENT widened from [A-Za-z0-9_-] to [A-Za-z0-9._:-] so
    natural credential names like "openai.prod" and "scope:billing"
    are accepted. Still rejects path separators and whitespace.

5 new tests + 1 rewritten test. Existing tests pass.
sroussey added a commit that referenced this pull request Jun 4, 2026
…ore (sec) (#544)

* fix(storage): harden ServerCredentialStore (key injection, delete races, orphan reasons)

Three classes of issue addressed:

1. Key-injection scope escape.
   userId, projectId, and every credential key are now matched against a
   strict SAFE_SEGMENT grammar (^[A-Za-z0-9_-]{1,128}$). Without this, a key
   like "../../other-user/other-project/leaked" could be smuggled through
   vaultId() and collide with another scope's vault id. Failures are surfaced
   as TypeError without echoing the invalid value back.

2. Delete races and orphan-overwrite.
   - delete() is refactored onto an internal deleteById() helper that
     deletes metadata FIRST, then vault. A throw on metadata.delete leaves
     the entry fully intact and readable; a throw on vault.deleteSecret
     reinstates a pending orphan marker (with orphanReason
     "vault-delete-failed") so readers see absence and operators can
     reconcile, AND surfaces a wrapped error whose message distinguishes
     "orphan marker persisted" from "orphan marker also failed to persist".
   - deleteById skips rows whose metadata is pending: true. This closes
     both the orphan-overwrite path (delete on an existing orphan marker
     used to nuke the vault entry it was tracking) and the concurrent
     put(new)+delete race (delete during the pending window of a new-entry
     put would wipe the vault entry mid-write).
   - get()/has() wrap expiry self-eviction in try/catch so a transient
     KV/vault failure during eviction does not surface as a thrown read.
   - deleteAll() collects per-row failures and throws AggregateError so a
     single bad row no longer silently masks deletion of the rest.

3. Orphan reason discriminator.
   CredentialMetadataRow gains an optional orphanReason field
   ("vault-write-failed" | "metadata-commit-failed" | "vault-delete-failed")
   and a corresponding exported OrphanReason type. Both put() orphan
   paths and the new delete() orphan path stamp the reason so operator
   tooling can pick a remediation strategy without re-running the original
   write. Legacy rows without the discriminator stay invisible to readers
   (covered by an explicit test).

14 new tests added; existing delete-removes-both test upgraded to assert
metadata-first ordering via spy call-order; existing put orphan tests
upgraded to assert orphanReason.

Supersedes #541.

https://claude.ai/code/session_01Wws8oZpB5imjKL2e7DRXtc

* fix(storage): restore ICredentialStore contract on ServerCredentialStore (sec)

Follow-up to #543. Two contract regressions discovered in code review:

H1: deleteAll() silently skipped pending/orphan rows because it now
    delegates to deleteById (added in #543) which short-circuits when
    pending=true. Result: no public API cleared orphan markers; a
    "wipe scope" call silently leaked the very rows operators need
    to clean up. Adds forceDeleteById that bypasses the pending-skip;
    deleteAll uses it. deleteById (single-key) unchanged.

H2: get/has/delete now threw TypeError for invalid keys because they
    all routed through vaultId(). ICredentialStore documents these as
    returning undefined/false on missing keys; throwing breaks
    substitutability with InMemoryCredentialStore / EncryptedKvCredentialStore.
    Adds isSafeKey() predicate; readers short-circuit; put() still
    throws via vaultId (only path that persists a colliding id).

M1: SAFE_SEGMENT widened from [A-Za-z0-9_-] to [A-Za-z0-9._:-] so
    natural credential names like "openai.prod" and "scope:billing"
    are accepted. Still rejects path separators and whitespace.

5 new tests + 1 rewritten test. Existing tests pass.

* docs(storage): clarify deleteById JSDoc + fix deleteByid typo (review)

Copilot review feedback on #544:

- deleteById's JSDoc claimed both delete() and deleteAll() share its
  code path, but deleteAll() deliberately routes through
  forceDeleteById() to bypass the pending-row short-circuit and
  reach sticky orphan markers. Updated to call out the asymmetry
  with a forward reference to forceDeleteById.
- Typo: 'deleteByid' -> 'deleteById' so the @link symbol resolves.

* refactor(storage): extract commitDelete to dedupe ServerCredentialStore delete paths

`deleteById` and `forceDeleteById` had nearly-identical bodies after the
pending-row check — the metadata-first ordering, orphan-marker rewrite,
and wrapped-error throw were duplicated, so a fix in one could silently
drift from the other. Lift the shared tail into `commitDelete(id, key,
existing, op)` so both callers route through one source of truth and
the only meaningful divergence (pending-row policy) stays at the call
site. Also corrects the stale `vaultId` JSDoc that still listed the
reader methods as callers — they were moved to `isSafeKey` in the same
PR for the ICredentialStore-contract substitutability fix.

https://claude.ai/code/session_011KMd9sERp2rguyekAi8a3u

---------

Co-authored-by: Claude <noreply@anthropic.com>
sroussey added 2 commits June 3, 2026 18:38
…w IPv6/redirect tests

PR #542 (Plan A initial-URL hardening) and PR #544 (Plan B
ServerCredentialStore hardening) landed on main and superseded both
commits in this branch. Reset the three production files to main's
versions and keep only the 4 IPv6 / redirect-canonicalisation tests
that main does not yet have.

Files reset to match main:
  - packages/ai/src/provider-utils/localOnlyFetch.ts          (#542)
  - packages/storage/src/credentials/ServerCredentialStore.ts (#544)
  - packages/storage/src/credentials/ServerCredentialStore.test.ts (#544)

File retaining net additions (4 new tests):
  - packages/test/src/test/ai-provider-api/localOnlyFetch.test.ts
      - accepts bracketed IPv6 loopback [::1] (positive)
      - rejects IPv6 zone identifier [::1%25eth0]
      - pins redirect behaviour: Location: http://0x7f.0.0.1/ from a
        loopback origin is followed because WHATWG canonicalises the
        target to 127.0.0.1
      - accepts bracketed IPv6 in a redirect Location

Verified locally on the rebased branch with `vitest run` against both
test files: 19/19 localOnlyFetch + 31/31 ServerCredentialStore pass.
…tch tests

Continuation of the previous rebase commit. Resets the two
ServerCredentialStore files to main's versions (PR #544 hardening
landed there) and updates the localOnlyFetch test file to main's
content plus the 4 new IPv6 / redirect-canonicalisation tests.

After this commit the PR diff against main is exactly:
  - +4 tests in packages/test/src/test/ai-provider-api/localOnlyFetch.test.ts
@sroussey sroussey changed the title fix(ai,storage): close localOnlyFetch redirect SSRF + harden ServerCredentialStore (sec) test(ai): add IPv6 + redirect-canonicalisation localOnlyFetch tests Jun 4, 2026
sroussey added 4 commits June 4, 2026 07:53
… cases

Reset `localOnlyFetch.test.ts` to main's content (which already includes the
three IPv4 canonicalisation tests landed by #542) and append four net-new
regression cases that main does not yet cover:

- accepts a bracketed IPv6 loopback initial URL (http://[::1]:8080/)
- rejects an IPv6 loopback with a zone ID (http://[::1%25eth0]/)
- follows a redirect whose Location canonicalizes to a loopback IPv4
  (extractRawHost(next.href) returns the canonical hostname; documented
  inline as the deliberate trade-off — the loopback-host invariant is
  preserved because the final destination IS 127.0.0.1)
- follows a redirect to a bracketed IPv6 loopback

This is the only conflict file between the branch and main; merging main
into this branch (via the GitHub "Update branch" action that follows this
commit) resolves all other files cleanly because every production file on
this branch already matches main exactly after PRs #542 and #544 landed.

https://claude.ai/code/session_01Wws8oZpB5imjKL2e7DRXtc
This commit resets the test file to byte-for-byte match main's version so
the upcoming "Update branch" merge has zero conflict. The four new IPv6 /
redirect-canonicalisation tests will be added as a separate follow-up
commit after the merge, on top of current main.

https://claude.ai/code/session_01Wws8oZpB5imjKL2e7DRXtc
…yFetch

Adds four cases not covered by main's localOnlyFetch suite after #542:
- accepts bracketed IPv6 loopback initial URL (http://[::1]:8080/)
- rejects IPv6 with zone ID (http://[::1%25eth0]/)
- follows redirect whose Location canonicalizes to 127.0.0.1
  (extractRawHost(next.href) returns the canonical hostname; documented
  inline as the deliberate trade-off — the loopback host invariant is
  preserved)
- follows redirect to bracketed IPv6 loopback

https://claude.ai/code/session_01Wws8oZpB5imjKL2e7DRXtc
@sroussey sroussey merged commit 25db668 into main Jun 4, 2026
13 checks passed
@sroussey sroussey deleted the claude/beautiful-mayer-gdhUE branch June 4, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant